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Binnen  het  verkennend  wetenschappelijk  onderzoeksprogramma  ‘de  Digitale  Werkomgeving’  bij 
TNO  Technische  Menskunde  wordt  bestudeerd  welke  mogelijkheden  Virtual  Environments  (VE) 
bieden  voor  hoogwaardige  interactie.  Bij  hoogwaardige  interactie  kunnen  gebruikers  niet  alleen 
virtueel  bewegen  en  rondkijken  en  iets  aanwijzen  in  een  gesimuleerde  wereld,  maar  kunnen  zij  ook 
op  natuurlijke  wijze  virtuele  voorwerpen  hanteren  en  manipuleren  (oppakken,  verplaatsen,  draaien, 
etc.).  Het  succes  van  geavanceerde  toepassingen  van  VE  op  interactief  ontwerp,  training  en  genees- 
kunde  is  sterk  afhankelijk  van  de  kwaliteit  en  vaardigheid  (snelheid,  nauwkeurigheid)  waarmee 
gebruikers  in  VE  hun  omgeving  kunnen  manipuleren. 

In  voorgaand  onderzoek  (Werkhoven  &  Groen,  1997)  heeft  TM  reeds  aangetoond  dat  directe 
manipulatietechnieken  (het  op  natuurlijke  wijze  met  een  virtuele  hand  grijpen  en  verplaatsen  van 
voorwerpen)  sneller  en  nauwkeuriger  zijn  dan  traditionele  indirecte  manipulatie  (via  muis/cursor 
interactie).  In  het  huidige  onderzoek  is  bestudeerd  of  taakprestaties  verminderen  wanneer  de  positie 
van  de  virtuele  hand  niet  precies  correspondeert  met  die  van  de  echte  hand  en  of  het  werken  met  een 
niet  corresponderende  virtuele  hand  tijdens  onderdompeling  in  VE  de  oog-hand  codrdinatie  bij 
terugkeer  in  de  werkelijke  wereld  beinvloedt. 

Er  is  een  prisma-adaptatie  paradigma  gebruikt  om  de  na-effecten  van  de  adaptatie  van  oog-hand 
codrdinatie  in  VE  te  kwantificeren.  Er  zijn  pre-  en  post-testen  uitgevoerd  om  de  aanwijsprestaties  in 
de  werkelijke  wereld  te  meten.  Tussen  pre-  en  post-testen  werden  participanten  ondergedompeld  in 
een  interactieve  virtuele  omgeving  waarin  de  virtuele  handpositie  opzettelijk  verschoven  was  over 
een  afstand  van  10  cm  ten  opzichte  van  de  echte  handpositie.  Door  de  resultaten  van  de  pre-  en 
post-test  te  vergelijken  werden  mogelijke  (negatieve)  na-effecten  van  de  adaptatie  van  de  oog-hand 
codrdinatie  aan  het  licht  gebracht.  Na-effecten  werden  gemeten  in  het  horizontal,  verticale  en 
diepte-vlak  van  de  virtuele  ruimte  voor  zowel  monoscopische  als  stereoscopische  kijk-condities. 

De  resultaten  laten  zien  dat  het  na-effect  in  laterale  richting  een  magnitude  heeft  van  20%  in  de 
richting  tegenovergesteld  aan  de  opzettelijke  verplaatsing  van  de  virtuele  hand  (een  negatief  na- 
effect).  Tijdens  het  manipuleren  in  de  VE  werd  geen  invloed  gevonden  van  correspondentie  of  niet- 
correspondentie  van  de  virtuele  met  de  echte  hand  op  de  prestatie. 

Het  geobserveerde  negatieve  na-effect  is  evidentie  voor  een  low-level  parameter  aanpassing  en  niet 
een  high-level  strategische  aanpassing  van  de  oog-hand  codrdinatie.  Daaruit  mag  geconcludeerd 
worden  dat  bij  trainingstoepassingen  aangeleerde  vaardigheden  in  VE  (ook  bij  niet  corresponde¬ 
rende  virtuele  hand)  overgedragen  zullen  worden  naar  de  werkelijke  wereld. 
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SUMMARY 

Virtual  hand  control  is  a  direct  natural  manipulation  method  in  virtual  environments  enabling 
advanced  applications  in  the  field  of  interactive  design,  training,  medicine,  etc.  TNO 
researchers  (Werkhoven  &  Groen,  1997)  have  shown  that  for  grasping  and  positioning  tasks 
virtual  hand  control  is  faster  and  more  accurate  than  traditional  mouse/cursor  interactions. 
However,  in  virtual  environments  the  virtual  hand  may  not  always  be  exactly  aligned  with  the 
real  hand.  Such  misalignment  may  cause  an  adaptation  of  the  users’  eye-hand  coordination. 
Further,  misalignment  may  cause  a  decrease  in  manipulation  performance  compared  to 
aligned  conditions. 

This  experimental  study  uses  a  prism  adaptation  paradigm  to  explore  visuo-motor  adaptation 
to  misaligned  virtual  hand  position. 

Participants  were  immersed  in  an  interactive  virmal  environment  with  a  deliberately  mis¬ 
aligned  virmal  hand  position  (a  lateral  shift  of  10  cm).  We  carried  out  pointing  tests  with  a 
non-visible  hand  in  the  real  world  before  (pre-test)  and  after  (post-test)  immersion  in  the 
virtual  world. 

A  comparison  of  pre-  and  post-tests  revealed  after-effects  of  the  adaptation  of  eye-hand 
coordination  in  the  opposite  direction  of  the  lateral  shift  (negative  after-effects).  The 
magnitude  of  the  after-effect  was  20%  under  stereoscopic  viewing  conditions.  However, 
decreased  manipulation  performance  in  VE  (speed/accuracy)  during  the  immersion  with 
misaligned  hand  conditions  was  not  found. 

The  occurrence  of  negative  after-effects  in  lateral  direction  indicates  that  adaptation  is  not 
explained  by  a  strategic  change  of  eye-hand  coordination  but  by  a  lower-level  parameter 
adjustment.  Therefore,  acquired  visuo-motor  skills  in  VE  are  likely  to  transfer  to  the  real 
world. 
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SAMENVATTING 

Het  op  directe,  natuurlijke  wijze  met  een  virtuele  hand  kunnen  manipuleren  van  virtuele 
omgevingen  maakt  geavanceerde  toepassingen  mogelijk  op  het  gebied  van  interactief  ont- 
werpen,  training,  geneeskunde,  etc.  TNO  onderzoekers  (Werkhoven  &  Groen,  1997)  hebben 
aangetoond  dat  voor  grijp-  en  positioneringstaken  het  manipuleren  met  een  virtuele  hand 
sneller  en  nauwkeuriger  is  dan  met  traditionele  muis/cursor  interactie.  Echter,  de  virtuele 
hand  in  virtuele  omgevingen  correspondeert  qua  positie  niet  altijd  met  de  echte  hand.  Derge- 
lijke  discrepanties  kunnen  een  adaptatie  veroorzaken  van  de  oog-hand  coordinatie  van  gebrui- 
kers.  Tevens  kan  het  zijn  dat  de  manipulatieprestatie  afneemt  ten  opzichte  van  een  conditie 
waar  de  positie  van  de  hand  wel  klopt. 

In  deze  experimentele  studie  is  een  prisma-adaptatie  paradigma  gebruikt  om  visuo-motor 
adaptatie  aan  niet  kloppende  virtuele  handposities  te  onderzoeken.  Er  zijn  pre-  en  post-testen 
uitgevoerd  om  de  aanwijsprestaties  in  de  werkelijke  wereld  te  meten.  Tussen  pre-en  post- 
testen  werden  participanten  ondergedompeld  in  een  interactieve  virtuele  omgeving  waarin  de 
virtuele  handpositie  opzettelijk  verschoven  was  over  een  afstand  van  10  cm  ten  opzichte  van 
de  echte  handpositie.  Door  de  resultaten  van  de  pre-  en  post-test  te  vergelijken  werden 
mogelijke  (negatieve)  na-effecten  van  de  adaptatie  van  de  oog-hand  coordinatie  aan  het  licht 
gebracht.  Na-effecten  werden  gemeten  in  drie  verschillende  dimensies  van  de  virtuele  ruimte 
voor  zowel  monoscopische  als  stereoscopische  kijk-condities. 

De  resultaten  laten  zien  dat  het  na-effect  in  laterale  richting  een  magnitude  heeft  van  20%. 
Een  vermindering  van  de  manipulatieprestatie  ten  gevolge  van  een  niet-corresponderende 
virtuele  hand  in  de  VE  werd  niet  gevonden. 

Een  negatief  na-effect  in  de  laterale  richting  suggereert  data  de  adaptatie  niet  bestaat  uit  een 
strategische  aanpassing  van  de  oog-hand  coordinatie,  maar  uit  een  parameter  aanpassing  op 
laag  niveau.  Het  is  daarom  waarschijnlijk  dat  in  VE  aangeleerde  visuo-motorische  vaardighe- 
den  overgedragen  zullen  worden  naar  de  echte  wereld. 
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1  INTRODUCTION 

Virtual  Environments  (VE)  are  three  dimensional  computer  generated  environments  offering 
full  immersion  and  potentially  allowing  a  high  level  of  interaction.  Current  applications 
already  offer  interaction  consisting  of  looking  around  (head-movements)  and  navigation  (by 
mouse-like  devices).  Such  interaction  is  sufficient  for  the  evaluation  of  cars,  ships,  buildings 
or  cities  in  the  design  phase  with  respect  to  sight  and  lay-out  (e.g.  Werkhoven,  Post  &  Punte, 
1997). 

For  advanced  simulations,  however,  just  looking  around  is  not  enough.  Applications  of  VE  in 
the  field  of  training  (Loftin  &  Kennedy,  1995),  interactive  design,  medicine  or  tele-operation 
(Park  &  Sheridan,  1991)  require  intuitive  direct  interaction  techniques  for  grasping  actions  or 
for  controlling  instruments. 

Direct  interaction  means  virtual  hand  control.  That  is,  users  manipulate  their  virtual  environ¬ 
ment  through  a  virtual  hand  that  mimics  the  movements  of  their  real  hand.  With  this  virtual 
hand,  virtual  objects  can  be  grasped,  rotated  or  positioned  in  ways  similar  to  manipulation 
using  the  real  hand.  Such  interaction  techniques  are  natural  and  intuitive  and  have  proven  to 
yield  better  performance  than  indirect  manipulation  techniques  through  mouse/cursor  control. 
Werkhoven  and  Groen  (1997)  compared  direct  manipulation  performance  (virtual  hand 
control)  with  indirect  manipulation  performance  (mouse-cursor  control)  under  both  mono- 
scopic  and  stereoscopic  viewing  conditions  and  found  that  direct  manipulation  is  both  faster 
and  more  accurate  for  grasping  and  positioning  tasks. 

Direct  manipulation  performance,  however,  is  expected  to  rely  on  the  quality  of  mimicking 
the  real  hand.  The  perceived  position  of  a  virtual  hand,  for  example,  depends  on  the  proper¬ 
ties  of  the  optics  of  head-mounted  displays  and  also  on  the  accuracy  of  tracking  the  position  of 
the  real  hand.  This  study  deals  with  the  question  if  users  of  virtual  hands  can  percepUially 
adapt  to  a  discrepant  visual  position  of  the  virtual  hand,  as  is  often  the  case  in  virtual  environ¬ 
ments,  and  whether  manipulation  performance  is  affected  by  discrepant  hand  positions. 


1.1  Discrepant  virtual  hand  position 

Where  is  the  virtual  hand  perceived?  Position  information  of  our  real  hand  is  always  provided 
by  our  proprioceptive  system,  mediated  by  receptors  in  our  muscles  and  skin.  In  a  VE 
system,  visual  feedback  of  the  virtual  hand  is  presented  through  the  head-mounted  display 
(HMD).  The  visual  and  proprioceptive  information  can  be  discrepant  for  several  reasons. 
First  of  all,  the  settings  of  the  geometrical  projection  model  of  the  computer  can  be  incorrect. 
For  instance,  to  obtain  correct  stereoscopic  depth  perception,  the  projection  model  has  to  be 
adjusted  to  the  inter  pupillary  distance  of  the  user.  To  save  time,  this  parameter  is  often  set  to 
the  population’s  average,  which  causes  an  incorrect  perception  of  depth  in  most  cases.  Other 
causes  of  mismatches  can  be  visual  distortions  as  caused  by  the  optics  of  the  HMD  (Robinett 
&  Rolland,  1992),  or  distortions  caused  by  the  tracking  system  used  to  determine  the  position 
of  eyes  and  hand.  It  is  fair  to  say  that  in  low-end  VE  systems  a  discrepancy  is  common. 
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Another  possible  cause  of  non-correspondence  between  seen  and  felt  virtual  hand  is  the  lack 
of  oculo-motor  depth  cues  in  virtual  environments.  In  monoscopic  displays,  accommodation 
and  convergence  of  the  eyes  are  fixed  at  a  single  level,  usually  at  infinity.  In  stereoscopic 
displays,  the  usual  coupling  between  convergence  and  accommodation  is  disrupted  in  order  to 
fuse  the  two  images  and  benefit  from  binocular  disparity.  Mis-accommodation  is  believed  to 
cause  perceptual  distortion,  although  the  exact  nature  of  the  effect  is  still  unclear  (Roscoe, 
1991). 

1.2  Adaptation 

What  happens  if  two  sources  of  information  about  the  same  event  are  discrepant?  People 
usually  do  not  notice  discrepancies  between  seen  and  felt  hand  position,  which  can  be  shown 
by  letting  people  wear  prisms,  which  displace  the  entire  visual  field  to  one  side.  The  visually 
displaced  hand  is  actually  felt  where  it  is  seen,  a  phenomenon  termed  immediate  visual 
capture  (Welch  &  Warren,  1980).  The  visual  sense  is  known  to  dominate  the  other  senses. 

Experiments  in  which  people  wear  prisms  for  longer  periods  show  that  eye-hand  coordination 
will  make  a  semi-permanent  adaptive  change  in  order  to  deal  with  the  new  relationship 
between  seen  and  felt  hand  position.  This  adaptive  change  leads  to  mis-reaching  to  the  oppo¬ 
site  side  of  the  prismatic  displacement  after  the  prisms  have  been  removed  (Helmholtz,  1910). 
The  difference  between  a  pointing  task  before  prism  exposure,  or  pre-test,  and  the  same  task 
directly  after  prism  exposure,  or  post-test,  is  called  the  (negative)  after-effect. 

Redding  and  Wallace  (1996)  theorize  that  prism  adaptation  is  a  remapping  between  internal 
representations  of  visual  space  and  proprioceptive  ‘limb  space’.  Limb  space  is  a  way  to 
describe  the  location  codes  used  for  open  loop  directional  movements  of  the  limbs.  Limb 
space  is  mapped  to  visual  space.  This  enables  us  to  reach  for  a  visual  target  without  having  to 
watch  our  hands  constantly.  When  the  relationship  between  visual  space  and  limb  space  is 
changed,  the  brain  is  capable  of  making  a  semi-permanent  adjustment  of  the  spatial  mapping 
function  that  links  both  spaces  (Bedford,  1989).  This  explains  the  short  term  negative  af¬ 
ter-effects  when  people  return  to  normal  conditions. 


1.3  Why  study  adaptation? 

There  are  several  reasons  why  a  study  of  adaptation  to  virtual  hands  is  important.  They  are 
discussed  below. 

1.3.1  Transfer  of  learning 

One  reason  to  study  adaptation  to  virtual  hands  is  the  significance  for  the  learning  and  transfer 
of  visuo-motor  skills.  Much  is  speculated  about  using  virtual  environments  for  learning 
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purposes,  such  as  surgery  (McGovern  &  Johnston,  1996)  or  military  operations  (Smith, 
1994).  Virtual  hands  or  other  body  parts  could  be  used  to  practice  visuo-motor  skills.  Skilled 
behavior  is  characterized  by  open-loop  control  and  replacing  visual  feedback  by  propriocep¬ 
tive  feedback  (Jordan  &  Rosenbaum,  1989).  According  to  the  theory  of  motor  skills,  motor 
programs  are  stored  in  the  brain  in  a  general  form.  At  a  lower  level,  parameters  are  filled  in 
to  fine  tune  the  motor  program  to  the  needs  of  the  specific  situation  (Schmidt,  1988). 

When  the  seen  virtual  hand  is  in  the  exact  same  position  as  the  felt  real  hand  and  there  is  no 
time  delay  between  hand  movement  and  the  corresponding  movement  of  the  virtual  hand, 
transfer  of  learning  should  be  possible,  because  motor  programs  will  correspond.  Unfortu¬ 
nately,  time  delay  and  spatial  discrepancies  are  inevitable  in  current  VE  systems  (Held  & 
Durlach,  1991). 

When  people  can  completely  adapt  to  a  discrepant  virtual  hand  position,  the  only  difference 
with  a  perfectly  calibrated  situation  is  the  adaptive  spatial  re-alignment  of  the  brain.  Such 
re-alignment  is  believed  to  be  obtained  by  lower  level  parameter  adjustment  (Bedford,  1993). 
This  implies  that  higher  level  motor  programs,  achieved  during  immersion  in  a  virtual  envi¬ 
ronment,  are  the  same  as  when  motor  learning  would  have  taken  place  in  a  real  environment. 
This  is  a  necessary  condition  for  visuo-motor  skills  learned  in  VE  to  be  preserved  and  suc¬ 
cessfully  applied  to  real  world  situations  or,  vice  versa,  for  skills  learned  in  the  real  world  to 
be  used  for  operation  in  a  vimial  environment. 

If  no  adaptation  takes  place,  the  brain  has  to  use  a  different  strategy  to  solve  the  discrepancy, 
which  can  be  referred  to  as  strategic  motor  control  (Redding  &  Wallace,  1996).  In  this  case, 
people  have  to  develop  specific  motor  programs  for  operating  in  the  virtual  environment,  and 
transfer  of  motor  skills  is  not  likely. 

•  Thus,  knowledge  about  the  conditions  needed  for  adaptation  in  VE  systems  can  help 
evaluate  learning  environments  and  make  predictions  about  transfer  of  motor  skills. 

1.3.2  Simulation  Fidelity 

There  is  another  reason  why  measures  of  adaptation  in  VE  are  useful.  Nemire,  Jacoby  and 
Ellis  (1994)  define  simulation  fidelity  as  the  extent  to  which  responses  to  changed  spatial 
relationships  in  the  simulated  environment  resemble  responses  to  analogous  changes  in  the 
real  environment.  For  instance,  a  lateral  shift  of  the  position  of  the  virtual  hand  can  be 
interpreted  as  a  simulation  of  wearing  wedge  prisms  in  the  real  environment.  From  this  point 
of  view,  adaptation  to  discrepant  virtual  hand  position  is  an  indicator  of  the  simulation  fidelity 
of  the  VE  system,  more  specifically  of  the  virtual  hand.  At  first  sight,  performance  tasks  seem 
to  be  the  most  appropriate  way  to  evaluate  the  display  characteristics  of  the  simulation.  The 
problem  is  that  performance  is  typically  influenced  by  experience  and  practice.  Furthermore, 
it  is  often  unclear  how  performance  of  a  certain  task  generalizes  to  other  domains. 
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•  Adaptation  after-effects  provide  an  objective  and  more  general  measure  for  evaluating  the 
simulation  fidelity  of  interactive  virtual  environments. 

1.3.3  Plasticity  of  the  brain 

Rolland,  Biocca,  Barlow  &  Kancherla  (1995)  argue  that  the  knowledge  about  the  plasticity  of 
the  brain  can  be  useful  for  manufacturers  of  VE  systems.  The  limitations  of  technology  will 
always  cause  some  form  of  intersensory  conflict. 

•  Knowledge  about  how  people  adapt  to  intersensory  discrepancies  can  guide  technology 
developments. 

1.3.4  After-effects 

Adaptation  leads  to  after-effects  in  eye-hand  coordination  when  we  return  to  the  real  world.  In 
some  cases  this  can  be  hazardous,  for  instance  when  head  mounted  displays  are  used  by 
surgeons. 

•  Knowledge  about  the  persistence  of  after-effects  is  needed  for  defining  safety  protocols. 


1.4  Can  we  adapt  to  our  virtual  hands? 

Can  people  adapt  to  a  virtual  hand,  as  observed  through  a  head  mounted  display?  In  current 
systems  virtual  space  is  not  as  well  specified  as  real  space,  due  to  limited  field  of  view,  low 
resolution,  slow  update  rate,  lack  of  valid  oculomotor  depth  cues  and,  in  monoscopic  dis¬ 
plays,  disparity.  Furthermore,  the  computer  model  of  the  hand  looks  different  than  the  real 
hand,  which  could  prevent  people  from  identifying  it  with  their  own  hand. 

After-effects  have  been  found  while  the  displaced  hand  was  represented  by  a  luminous  dot  in 
an  otherwise  dark  room  (Welch,  1972).  Even  if  participants  were  aware  that  the  dot  was 
attached  to  the  experimenter’s  hand,  and  not  to  their  own,  adaptation  after-effects  occurred, 
though  somewhat  less  than  when  participants  were  unaware  of  this.  Apparently,  adaptation  is 
not  much  affected  by  low  resolution  or  people’s  awareness  of  the  source  of  visual  input  from 
the  hand. 

Rolland  et  al.  (1995)  measured  after-effects  after  seeing  the  world  mediated  by  video  cam¬ 
eras,  attached  to  a  low  resolution  stereoscopic  head  mounted  display.  This  augmented  reality 
system  caused  the  virtual  eye-position  to  be  about  6  cm  higher  and  16  cm  away  from  the 
operators’  own  eyes.  This  caused  their  hands  to  be  seen  6  cm  lower  and  16  cm  closer.  Small 
negative  after-effects  were  found  in  both  directions.  This  is  promising  for  adaptation  in 
computer  simulated  environments,  using  similar  displays. 
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1.5  Time  delay 

Of  concern  is  the  considerable  time  delay  between  hand  and  head  movements  and  the  render¬ 
ing  of  the  corresponding  image,  which  characterizes  current  VE  systems.  The  delay  of  head 
movements  further  degrades  the  perception  of  the  spatial  layout  of  the  virtual  environment. 
The  delay  of  virtual  hand  movements  also  raises  questions  to  the  possibility  of  adaptation. 
Held,  Efstathiou  and  Greene  (1966)  showed  that  adaptation  to  prismatic  displacement  is 
totally  blocked  by  time  delays  of  300  ms.  Held  and  Durlach  (1991)  found  that  prism  adapta¬ 
tion  is  heavily  reduced  when  time  delays  exceed  60  ms,  which  is  not  unusual  in  most  VE 
systems.  Given  the  poor  specification  of  space  in  combination  with  time  delay,  it  is  not  clear 
if  people  can  adapt  to  virtual  hand  position  in  a  current  VE  system. 


1.6  This  experiment 

The  purpose  of  the  present  experiment  was  to  explore  if  the  brain  makes  an  adaptive 
re-alignment  of  visual  and  limb  space  to  deal  with  a  discrepancy  between  real  and  virtual  hand 
while  doing  a  manipulation  task  in  a  current  virtual  environment  system,  that  is,  using  a  low 
resolution  head  mounted  display  under  a  time  delay  of  more  than  60  ms.  The  only  way  to  be 
sure  that  adaptive  re-alignment  has  taken  place  is  to  show  negative  after-effects  in  open  loop 
limb  positioning  (Redding  &  Wallace,  1996;  Welch,  1978).  Monoscopic  and  stereoscopic 
conditions  were  compared  to  see  if  a  stereoscopic  image  can  help  adaptive  re-alignment. 

1.6.1  Subjective  calibration  of  the  virtual  workspace 

Since  negative  after-effects  are  found  in  relation  to  the  discrepancies  between  seen  and  felt 
hand,  the  nature  of  the  discrepancies  inherent  to  our  VE  system  had  to  be  known  first. 
Theoretically,  it  should  be  possible  to  derive  the  visual  distortion  from  the  physical  properties 
of  the  VE  system  used.  The  problem  is  that  no  complete  model  exists,  which  takes  all  aspects 
contributing  to  visual  position  information  into  account.  Furthermore,  up  to  now  it  is  unclear 
how  discrepant  depth  cues  are  handled  by  our  perceptual  system. 

A  more  convenient  way  therefore,  is  to  use  a  subjective  method  as  proposed  by  Ellis  & 
Nemire  (1993),  where  people  subjectively  calibrate  virtual  space  by  pointing  to  virtual 
markers  with  their  unseen  hand.  In  this  way  a  rough  linear  estimation  of  the  perceptual 
distortion  of  viitual  space  can  be  made.  Mismatches  in  the  depth  dimension,  that  is  perpendic¬ 
ular  to  the  screen  surface,  are  expected  due  to  conflicting  depth  cues  as  described  above  and 
inaccuracies  of  the  head  mounted  display  used.  In  general,  perceiving  the  depth  dimension  in 
displays  is  more  difficult  than  other  dimensions  and  prone  to  errors  (Zhai  &  Milgram,  1994). 
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1.6.2  Lateral  shift  of  the  virtual  hand 

On  top  of  the  discrepancies  caused  by  VE  system  inaccuracies  itself  (optics  and  tracking),  the 
virtual  hand  was  shifted  10  cm  to  the  left  or  right  to  create  an  unambiguous  difference  be¬ 
tween  seen  and  felt  hand  position  in  the  lateral  direction.  This  was  done  in  order  to  create  a 
possible  after-effect  big  enough  to  exceed  the  noise  of  the  measuring  technique.  A  lateral  shift 
allows  the  results  to  be  linked  to  the  large  body  of  prism  adaptation  literamre  which  focuses 
on  lateral  displacements  of  the  visual  field  (Kornheiser,  1976;  Welch,  1978).  It  also  enables 
us  to  compare  the  simulation  fidelity  of  monoscopic  and  stereoscopic  virtual  environments. 

1.6.3  Comparison  of  performance  between  shifted  hand  and  aligned  hand 

Mismatches  between  virtual  and  real  hand  position  are  believed  to  degrade  performance 
(Ellis,  1991;  Rolland  et  al.,  1995).  This  hypothesis,  however,  has  never  really  been  tested. 
Good  calibration  of  the  virtual  hand,  that  is  a  good  match  of  virtual  and  real  space,  is  time 
consuming  and  therefore  expensive.  The  manipulation  task  used  for  the  adaptation  experiment 
was  the  same  as  for  the  manipulation  experiment  by  Werkhoven  and  Groen  (1997),  except  for 
the  right  or  left  shift  of  the  virtual  hand.  This  makes  it  possible  to  compare  the  performance  of 
a  displaced  virtual  hand  with  an  aligned  virtual  hand  and  might  shed  some  light  on  the  bene¬ 
fits  of  a  well-calibrated  virtual  hand. 


2  METHOD 

The  experiments  consist  of  a  pre-test,  exposure  to  a  manipulation  task,  a  post-test  and  a 
subjective  calibration-test. 

The  pre-test  is  carried  out  to  get  a  base-line  for  the  pointing  performance  to  real  targets  in  the 
real  world.  The  hand  was  not  visible  during  pointing.  Results  of  the  pre-test  are  compared 
with  the  results  of  the  post-test  (identical  to  the  pre-test)  that  is  carried  out  after  immersion 
and  manipulation  in  VE  with  a  deliberately  misaligned  hand.  Differences  in  the  results  show 
how  our  eye-hand  coordination  has  adapted  to  the  new  conditions  of  the  virtual  environment 
(see  Figure  1). 

To  let  participants  adapt  to  the  virmal  environment  they  carried  out  manipulation  tasks  in  VE. 
Participants  were  instructed  to  grasp  virtual  objects  (cubes),  to  rotate  them  and  to  place  them 
on  a  position  defined  by  other  cubes. 
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eye-hand  coordination  before  immers 

on  in  VE 

Fig.  1  This  sketch  illustrates  the  functions  of  the  pre-test  in  the  real  world,  the  expo¬ 
sure  to  new  conditions  in  the  virtual  environment,  and  the  post-test  in  the  real  world  to 
reveal  adaptation  effects  (after-effects)  of  eye-hand  coordination. 


Finally,  a  subjective  calibration-test  (so  called  by  Ellis  &  Nemire,  1993)  was  carried  out  to 
reveal  perceptual  deformations  inherent  to  our  VE  system.  Here  participants  point  to  the 
position  of  virtual  objects  in  VE  with  their  real  hand,  while  both  the  real  and  virtual  hand  are 
invisible.  Ellis  suggests  comparing  the  pointing  functions  in  VE  (the  subjective  calibration- 
test)  with  functions  obtained  from  pointing  at  real  targets  (the  pre-test).  In  this  way,  subjective 
errors  in  open-looped  target  pointing  and  systematic  errors  caused  by  the  pointing  technique 
can  be  filtered  out. 


2.1  Participants 

Eight  right  or  both-handed  students  participated  in  the  experiment.  Their  age  ranged  from  17 
to  27.  They  all  had  normal  vision,  or  corrected  to  normal  by  contact  lenses. 
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2.2  Apparatus 

Image  generation.  Computer  graphics  were  generated  by  an  Onyx  Reality  Engine,  manufac¬ 
tured  by  Silicon  Graphics.  The  virmal  environment  was  drawn  in  normal  linear  perspective 
and  depth  cues  such  as  occlusion  and  shading  were  used.  The  refresh  as  well  as  update  rate 
was  60  Hz. 

Head-mounted  displays.  Images  were  presented  by  means  of  the  Eyegen  3™,  manufactured  by 
Virtual  Research  Ltd.  This  head  mounted  display  (HMD)  has  two  color  displays  with 
250x493  pixels.  The  effective  field  of  view  is  approximately  40°  diagonal,  resulting  in  a 
resolution  of  4.4  arc  minutes.  Viewing  the  outside  world  is  blocked  by  eyecups.  The  distance 
between  the  left  and  right  mini-display  of  the  HMD  can  be  adjusted  to  fit  the  user.  In  stereo¬ 
scopic  conditions,  the  interocular  distance  of  the  projection  model  was  set  to  60  mm.  In 
monoscopic  conditions,  it  was  set  to  zero  (‘Cyclopean  vision’).  It  should  be  noted  that  the 
monoscopic  viewing  condition  was  still  bi-ocular  (not  monocular!). 

Head  tracking.  Head  position  and  orientation  were  tracked  using  the  Fastrak™  (Polhemus),  an 
electromagnetic  position  and  orientation  sensing  device.  It  consists  of  one  transmitter  and  two 
receivers,  one  of  which  was  attached  to  the  head  mounted  display.  The  transmitter  was 
positioned  just  above  the  place  where  the  participants  were  seated.  The  range  within  which 
tracking  accuracy  is  less  than  3  mm  error  was  less  than  1  m.  The  Fastrak  and  Eyegen  3  are 
extensively  described  in  Werkhoven  and  Hoekstra  (1994). 

Hand  tracking.  We  used  a  manipulation  task  described  by  Werkhoven  and  Groen  (1997)  in  an 
immersive  virtual  environment.  For  hand  tracking  the  DataGlove™  was  used.  Position  and 
orientation  of  the  hand  were  tracked  by  the  second  receiver  of  the  Polhemus  tracking  device, 
mounted  on  the  back  of  a  DataGlove^“  (Fifth  Dimension  Ltd.)  and  worn  on  the  right  hand. 
The  glove  contains  fiber  optics,  and  remrns  for  each  finger  a  bending  parameter  at  120  Hz 
(Burdea  &  Coiffet,  1993).  The  position,  orientation  and  bending  parameters  were  used  to 
render  a  realistic  model  of  the  human  hand,  which  consisted  of  about  5,000  polygons. 

To  obtain  smooth  finger  movement,  at  each  cycle,  only  20%  of  the  difference  between  the 
value  of  the  actual  bending  of  the  real  and  visualized  (virmal)  finger  was  added  to  the  new 
value  of  the  bending  of  the  virtual  finger.  This  caused  an  additional  average  delay  for  finger 
bending  of  approximately  70  ms. 

A  virtual  object  could  be  ‘grasped’,  that  is  attached  to  the  hand,  by  making  contact  with  both 
thumb  and  index  finger  to  the  object  while  keeping  the  three  other  fingers  closed.  It  could  be 
released  by  opening  the  hand. 

For  half  of  the  participants,  the  virtual  hand  was  programmed  10  cm  to  the  left,  for  the  other 
half  it  was  programmed  10  cm  to  the  right. 

Data-collection.  The  data  streams  from  the  data-glove,  mouse  and  position  tracker  for  head 
and  hand  were  collected  and  processed  by  a  program  running  on  a  486DX/60  MHz  personal 
computer,  and  sent  to  the  Onyx.  The  total  delay  between  movements  of  the  head  or  hand  and 
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presentation  of  the  corresponding  image  was  approximately  80  ms.  A  second  personal  com¬ 
puter  controlled  the  experiment  and  recorded  completion  times  of  the  different  manipulation 
tasks  and  the  positions  and  orientations  of  the  virtual  cubes  while  they  were  attached  to  the 
virtual  hand  or  cursor. 

Set-up  for  pre-  and  post-tests.  For  the  pre-  and  post-tests  of  target  pointing,  a  construction  was 
used  inspired  on  a  method  used  by  Held  and  Gottlieb  (1958).  It  is  schematically  depicted  in 
Figure  2.  It  consists  of  a  wooden  box  (65  x  80  x  65  cm  hxwxd)  mounted  on  top  of  a 
wheeled  table.  A  flat  mirror  covered  the  bottom  of  the  wooden  box  and  a  board  was  attached 
to  an  aperture  in  the  frontal  side  of  the  box,  making  an  angle  of  45°  with  the  horizontal  plane. 
The  board  contained  a  small  viewing  hole,  and  an  indentation  for  the  nose.  The  inside  of  the 
wooden  box  was  illuminated  by  shining  a  spotlight  through  an  aperture  at  the  left  side  of  the 
box.  Five  wooden  cubes  with  edges  of  2  cm  could  be  positioned  above  the  mirror.  The  cubes 
were  attached  to  the  end  of  small  wooden  sticks.  The  sticks  could  be  pushed  back  and  forth 
through  little  holes  at  the  right  side  of  the  box  in  order  to  adjust  the  horizontal  position  of  the 
cubes. 


Fig.  2  Schematic  drawing  of  the  experimental  apparatus  used  for  pre-  and  post-tests. 


The  viewing  hole  offered  a  view  on  only  the  mirror  images  of  the  cubes  but  not  on  the  cubes 
themselves.  The  mirror  cubes  appeared  to  be  within  the  reaching  range  of  the  hand.  When 
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reaching,  the  view  of  the  hand  was  obstructed  by  the  mirror.  The  three  dimensional  position 
of  the  hand  was  recorded  by  using  the  Polhemus  sensor.  For  these  target  pointing  tasks,  the 
sensor  was  removed  from  the  back  of  the  DataGlove  and  held  between  the  thumb  and  index 
finger. 

Subjective  calibration.  For  the  subjective  calibration  of  virtual  space,  the  head  mounted 
display  could  be  attached  to  the  wooden  box  used  for  the  pre-  and  post-tests,  so  that  it  was 
immobile.  When  looking  through  it,  the  viewing  axis  made  an  angle  of  45°  with  the  horizon¬ 
tal  plane,  similar  to  the  view  on  the  mirror.  While  pointing  at  virtual  cubes,  the  virtual  hand 
was  programmed  to  be  invisible.  The  total  construction  could  be  rolled  to  the  left  in  order  to 
make  way  for  the  adaptation  task.  A  wooden  obstruction  fixed  to  the  ground  enabled  the 
experimenter  to  reposition  the  box  at  exactly  the  same  place  in  front  of  the  participant  for  the 
post-test. 


2.3  Stimuli 

Workspace.  The  virtual  workspace  consisted  of  a  desk  surface  with  telephone  and  bookshelf, 
surrounded  by  walls  (see  Figure  3).  All  surfaces  were  texture  mapped.  The  near  side  of  the 
desk  had  a  semi-circular  indentation,  in  the  middle  of  which  the  participant  was  seated,  facing 
the  desk.  Participants  were  seated  on  a  swivel-chair  on  wheels. 


Fig.  3  The  virtual  work  space. 


Manipulation.  The  stimuli  for  the  manipulation  task  used  for  adaptation  in  VE  were  similar  to 
the  task  described  by  Werkhoven  and  Groen  (1997). 

For  the  rotation  tasks,  participants  were  asked  to  grasp  a  die  presented  on  the  desk  at  the 
right-hand  side.  The  size  of  the  die  was  10  cm. 
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For  the  positioning  task  three  cubes  were  positioned  above  the  desk  surface.  They  served  as 
three  corners  of  an  imaginary  square  (defined  by  the  centers  of  the  three  cubes),  which  could 
be  either  parallel  to  the  desk  (in  the  horizontal  plane)  or  parallel  to  the  frontal  plane.  Partici¬ 
pants  were  asked  to  position  the  die  (size  10  cm)  on  the  fourth  (missing)  corner.  The  configu¬ 
ration  of  these  cubes  also  varied  over  trials.  For  both  orientations  of  the  square  (in  horizontal 
or  frontal  plane),  each  of  the  four  possible  corners  could  be  the  missing  one.  The  edge  size  of 
the  cubes  constituting  the  square  was  10  cm.  Distances  between  these  cubes  were  50  cm.  The 
8  possible  configurations  (four  corner  positions,  two  orientations  of  the  square)  were  pre¬ 
sented  once  in  random  order  for  each  combination  of  viewing  (mono/stereo)  and  manipulation 
(direct/indirect)  condition  tested.  The  center  of  the  square  was  randomly  assigned  to  one  of 
seven  possible  locations:  one  was  the  basic  location,  40  cm  above  the  desk,  50  cm  away  from 
the  observer,  and  shifted  20  cm  to  the  left.  The  other  six  could  be  calculated  by  adding  or 
subtracting  5  cm  along  one  of  the  three  axes. 

Pre-  and  post-tests.  For  the  target  pointing  tests  in  the  real  world  (pre-  and  post-tests), 
wooden  cubes  at  seven  different  target  positions  were  presented  five  times  each  in  random 
order.  Three  cubes  were  at  the  viewing  axis  at  distances  of  33,  43  and  53  cm  from  the 
viewing  hole.  Two  cubes  were  positioned  10  cm  perpendicular  below  and  above  the  center  of 
the  three  cubes  at  the  viewing  axis  and  two  cubes  12.5  cm  left  and  right  of  the  center  cube  (at 
the  horizontal  axis). 

Subjective  calibration.  For  the  calibration  of  virtual  space,  seven  virmal  cubes  with  5  cm 
edges  were  presented  five  times  in  random  order.  The  coordinates  of  the  virtual  cubes  relative 
to  the  eye  were  the  same  as  those  used  for  the  real  cubes  in  the  pre-  and  post-tests,  except  for 
the  three  cubes  along  the  horizontal  axis.  Their  original  positions  were  not  visible  within  the 
limited  field  of  view  of  the  head-mounted  display  used.  Therefore,  they  were  moved  7  cm 
further  along  the  viewing  axis,  and  7  cm  downward  along  the  vertical  axis  so  that  they 
became  visible.  Judgments  were  only  done  with  stereoscopic  vision  because  under  mono- 
scopic  conditions  a  lack  of  disparity  in  the  now  stationary  head  mounted  display  provided 
insufficient  depth  information  to  judge  the  distance  with  reasonable  confidence. 


2.4  Design 

A  within-participants  design  was  used  with  four  conditions  that  were  combinations  of  the 
manipulation  condition  (left-  or  rightward  lateral  displacement  of  the  virtual  hand)  and  view¬ 
ing  condition  (stereoscopic  or  monoscopic).  The  order  of  conditions  was  balanced  (see 
Table  I).  The  dependent  variable  was  the  negative  after-effect  along  the  horizontal,  vertical 
and  viewing  axis.  The  negative  after-effect  is  defined  by  the  difference  between  hand  position 
in  pre-  and  post-test,  in  the  opposite  direction  of  the  visual  displacement  of  the  hand.  In  each 
group,  the  order  of  conditions  was  balanced  (see  Table  I). 
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Adaptation  was  tested  for  each  participant  for  each  viewing  condition  (mono  and  stereo)  and  a 
single  displacement  (left  or  right).  The  adaptation  session  consisted  of  35  positioning  trials. 
Table  I  shows  the  sequence  of  sessions  for  each  participant  (SL= stereo/left  shift,  ML== 
mono/left  shift,  SR = stereo/right  shift,  MR = mono/right  shift). 


Table  1  Experimental  design. 


1st 

2nd 

participant 

adaptation 

session 

adaptation 

session 
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1 
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SL 
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2 
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ML 
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B 

R 
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E 

A 

T 

5 

S 

SL 

E 

S 

A 
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S 

ML 

E 

S 

A 

K 

I 

O 

N 

6 

T 

SR 

T 

T 

MR 

T 

7 

ML 

SL 

8 

MR 

SR 

SL:  stereo,  left  shift;  SR:  stereo,  right  shift 
ML:  mono,  left  shift;  MR:  mono,  right  shift 


2.5  Procedure 

General.  Participants  read  instructions  about  the  working  of  the  VE  system.  They  were  told 
that  their  eye-hand  coordination  was  tested  before  and  after  the  adaptation  phase  (immersion 
in  VE).  They  were  seated  on  a  desk  chair  in  front  of  the  wooden  box  looking  through  the 
viewing  hole.  During  pre-  and  post-test,  participants  wore  the  DataGlove.  This  was  done  to 
assure  that  after-effects  could  not  be  due  to  adaptation  to  the  weight  of  the  DataGlove. 

Pre-test.  The  pre-test  went  as  follows.  The  position  sensor  was  held  between  thumb  and  index 
finger.  The  five  cubes  were  at  the  left  most  side  of  the  box.  The  experimenter  placed  a  cube 
at  one  of  the  target  positions  by  pushing  the  wooden  stick  forward.  Following  this,  partici¬ 
pants  placed  the  position  sensor  to  coincide  with  the  mirror  image  of  the  cube  and  gave  a 
verbal  sign  when  finished.  No  time  limit  was  given.  The  experimenter  recorded  the  sensor 
position  by  pressing  a  key.  After  this,  participants  withdrew  their  hand  and  the  cube  was 
pushed  back  to  its  initial  position.  In  this  way,  all  35  positions  were  tested. 

Manipulation  task.  After  the  pre-test  the  wooden  box  was  moved  away  and  the  position  sensor 
was  mounted  on  the  glove.  Participants  did  a  manipulation  task  in  the  virtual  environment, 
under  stereoscopic  as  well  as  monoscopic  conditions.  The  manipulation  tasks  were  similar  to 
the  tasks  used  by  Werkhoven  and  Groen  (1997). 
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At  the  beginning  of  the  trial,  a  die  was  presented  on  the  desk  on  either  the  left-hand  or  the 
right-hand  side  of  the  participant.  When  an  audible  tone  sounded,  they  grasped  the  die.  When 
the  die  was  presented  on  the  left-hand  side,  participants  were  asked  to  position  the  die  on  the 
missing  corner  of  an  imaginary  square  formed  by  the  middle  points  of  the  cubes  (see  Stimuli). 
The  orientation  of  the  die  had  to  match  the  orientation  of  the  other  cubes  (the  edges  aligned 
with  the  coordinate  axes  of  the  VE). 

When  the  die  was  presented  on  the  right-hand  side  participants  had  to  rotate  the  cube  follow¬ 
ing  a  specific  sequence.  After  they  had  grasped  it,  they  pitched  it  90°  until  number  one  was 
on  top.  When  the  surface  containing  the  number  one  made  an  angle  with  the  ground  plane  of 
less  then  10°,  a  second  tone  confirmed  that  the  pitch  was  successful.  After  this  tone,  the  dice 
had  to  be  rolled  90°  to  the  left  until  number  one  was  on  the  left  side  of  the  die  and  perpendic¬ 
ular  to  the  sagittal  plane.  Again  a  difference  of  10°  was  allowed.  After  completion  a  second 
tone  sounded,  and  the  die  could  be  released.  Participants  were  instructed  to  grasp  and  rotate 
the  dice  as  fast  as  possible. 

There  were  no  practice  trials.  The  grasping  and  releasing  of  cubes  was  demonstrated  by  the 
experimenter. 

Post-test.  Immediately  after  the  last  trial,  participants  were  asked  to  close  their  eyes.  This  was 
to  prevent  quick  decay  of  adaptation  when  viewing  the  real,  undistorted  hand.  The  table 
containing  the  wooden  box  was  moved  in  place.  The  experimenter  removed  the  head  mounted 
display  and  handed  the  position  sensor  to  the  participant.  After  the  participants  opened  their 
eyes,  the  experimenter  started  the  post-test,  which  was  identical  to  the  pre-test. 

Second  condition.  Two  hours  after  finishing  the  first  post-test  participants  were  tested  a 
second  time  under  the  other  viewing  condition  following  the  same  sequence  of  pre-test, 
adaptation  and  post-test. 

Subjective  calibration  test.  Fifteen  minutes  after  the  second  post-test,  the  calibration  test  was 
done.  Participants  looked  through  the  head  mounted  display,  which  was  now  fixed  to  make  an 
angle  of  45°  with  the  horizontal  plane.  This  time  35  virtual  cubes  were  presented  one  by  one. 
Participants  were  asked  to  position  the  position  sensor  on  the  place  were  they  saw  the  virtual 
cube.  They  verbally  reported  when  they  were  ready.  The  cube  disappeared  after  the  experi¬ 
menter  recorded  the  sensor  coordinates.  After  the  hand  was  withdrawn,  the  next  cube  was 
presented. 


2.6  Analysis 

The  positioning  results  are  measured  along  axes  that  were  aligned  with  the  virtual  desk. 
However,  for  the  analysis  of  the  results  of  the  pre-,  post-  and  subjective  calibration  tests  we 
wanted  to  express  the  positioning  results  along  dimensions  relative  to  the  (fixed)  viewing 
direction  of  the  participants.  That  is,  the  axis  along  the  viewing  direction  (called  depth),  the 
horizontal  axis  orthogonal  to  the  viewing  direction  and  the  axis  orthogonal  to  horizontal  and 
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depth  axis  (the  new  vertical  axis).  Basically  this  new  coordinate  system  is  the  old  system 
rotated  about  the  horizontal  axis  such  that  the  old  depth  axis  was  aligned  with  the  viewing 
direction  (the  new  depth  axis). 


3  RESULTS 

For  each  target  cube,  real  and  virtual,  the  mean  coordinate  of  five  pointing  trials  was  calcu¬ 
lated  for  each  position,  condition  and  participant.  When  the  standard  deviation  of  the  five 
pointing  results  exceeded  three  cm,  the  value  deviating  the  most  from  the  median  was  consid¬ 
ered  an  outlier. 


3.1  Subjective  calibration 

To  get  an  impression  of  the  perceptual  distortion  of  virtual  space,  the  average  position  of  the 
virtual  cubes  pointed  to  by  participants  was  plotted  against  the  position  used  by  the  projection 
model  of  the  computer  for  each  dimension  (horizontal,  vertical  and  depth).  A  linear  regres¬ 
sion  line  was  fitted.  This  was  done  separately  for  each  dimension.  The  slope  of  the  line 
reflects  a  first  order  linear  scaling  of  virtual  space.  When  no  distortion  is  present,  the  ex¬ 
pected  value  is  one.  A  larger  or  smaller  slope  corresponds  to  magnifications  or  minifications. 
The  intercept  is  expected  to  be  zero  cm.  Non-zero  values  of  the  intercept  indicate  a  constant 
shift  of  the  visual  space  perceived.  Note  that  the  subjective  calibration  test  was  carried  out 
under  stereoscopic  viewing  conditions  only. 

Regression  results  for  the  subjective  calibration  tests  (and  other  tests)  are  shown  in  Table  II.  A 
slope  value  marked  with  an  asterisk  indicates  that  this  slope  value  differs  significantly  from 
one  (p<0.05).  An  asterisk  marking  an  intercept  value  indicates  that  this  intercept  value 
differs  significantly  from  zero  (p<0.05). 

Depth  dimension.  The  line  labeled  ‘calibration’  in  Figure  4  is  the  regression  function  for 
pointing  results  along  the  viewing  axis  (depth).  The  regression  function  is  described  by  a 
slope  of  0.55  and  an  intercept  value  of  3.02  cm  with  a  squared  correlation  coefficient 
p2=0.86.  Obviously,  perceptual  deformation  occurs  because  the  regression  coefficient 
deviates  significantly  from  one.  To  distinguish  open-loop  pointing  errors  from  perceptual 
distortions  in  VE  we  will  compare  this  regression  line  with  pointing  results  in  the  real  world 
(the  pre-test)  in  the  next  section. 

Horizontal  dimension.  The  regression  line  in  the  horizontal  dimension  for  the  subjective 
calibration  test  (virtual  targets)  had  a  slope  of  0.85  and  an  intercept  value  of  -3.0  cm  with  a 
squared  correlation  coefficient  p2=0.99.The  slope  and  intercept  values  do  not  reflect  a 
perfectly  calibrated  situation  and  differ  significantly  from  one  and  zero,  respectively. 
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Fig.  4  Results  of  the  subjective  calibration  test  and  pre-test  for  the  depth  dimen¬ 
sion  (the  distance  along  the  viewing  direction).  The  depth  of  targets  indicated  by 
participants  is  plotted  vertically  against  the  actual  positions  of  real  targets  (pre-test) 
or  virtual  targets  (subjective  calibration  test).  The  point  between  the  eyes  was  taken 
as  the  origin.  Viewing  conditions  were  stereoscopic. 


Vertical  dimension.  In  the  vertical  dimension  regression  lines  for  the  subjective  calibration  test 
had  a  slope  of  0.95  and  an  intercept  of  -0.57  cm  with  a  squared  correlation  coefficient  p^= 
0.99.  The  deviations  of  slope  and  intercept  from  what  we  expect  in  perfectly  calibrated 
situations  are  not  significant. 


3.2  Pre-tests  and  a  comparison  with  the  snhjective  calibration  results 

This  section  shows  the  results  of  the  pre-tests  in  which  participants  pointed  to  real  targets 
before  immersion  in  VE.  To  facilitate  a  direct  comparison  with  the  results  of  the  post-tests 
under  stereo  conditions,  we  have  chosen  to  present  only  the  pre-tests  that  preceded  immersion 
under  stereo  conditions.  Nevertheless,  the  results  of  pre-tests  are  expected  to  be  independent 
of  the  viewing  condition  (mono/stereo)  during  immersion  in  VE.  Indeed  no  significant  differ¬ 
ences  have  been  observed  between  pre-tests  preceding  mono  and  stereo  conditions. 

To  facilitate  a  comparison  of  subjective  calibration,  pre-  and  post-tests,  regression  results 
(slopes,  intercepts  and  squared  correlation  coefficients)  for  all  tests  are  given  in  Table  II. 
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Table  II  Regression  results  for  subjective  calibration,  pre-  and  post-tests  (stereo-condi¬ 
tions  only). 


Dimension 

Subjective  calibration 

Pre-test 

Post-test 

S 

I 

s 

I 

S 

I 

P^ 

horizontal 

0.85* 

-3.00’ 

0.99 

0.93* 

-3.10* 

1.00 

L:  0.97 

-0.88* 

1.00 

R:  1.02 

-6.45* 

1.00 

depth 

0.55‘ 

3.02 

0.86 

1.07 

-4.29 

0.98 

1.10 

-1.58 

0.98 

vertical 

0.95 

-0.57 

0.99 

0.86* 

-1.88’ 

1.00 

0.93* 

-0.57* 

1.00 

S;  slope;  I:  intercept;  p^:  squared  correlation  coefficient; 

L;  left  virtual  hand  displacement;  R:  right  virtual  hand  displacement; 

A  slope  value  marked  with  an  asterisk  indicates  that  this  slope  value  differs  significantly  from  one  (p<0.05). 
An  asterisk  marking  an  intercept  value  indicates  that  this  intercept  value  differs  significantly  from  zero 
(p<0.05). 


Depth.  Figure  4  shows  the  regression  line  (labeled  ‘pre-test’)  for  the  positions  indicated  along 
the  viewing  direction  (depth)  versus  the  actual  position  of  the  (real)  targets  during  those  pre¬ 
tests  preceding  stereo  conditions.  This  regression  line  has  a  slope  of  1.07  and  an  intercept 
value  of  -4.29  cm  with  a  squared  correlation  coefficient  p2=0.98.  The  slope  of  this  regres¬ 
sion  line  does  not  differ  significantly  from  the  value  1  suggesting  that  open-loop  target  point¬ 
ing  errors  are  not  significant.  Therefore,  the  significant  difference  between  the  slopes  of  the 
pre-test  and  subjective  calibration  lines  [t(10)=4.15,  p< 0.005]  can  only  be  explained  by  the 
perceptual  distortion  of  the  virtual  environment.  The  significantly  smaller  slope  in  VE  sug¬ 
gests  an  underestimation  of  depth  of  almost  50%  in  VE. 

Horizontal  dimension.  For  the  horizontal  dimension,  the  regression  line  for  the  pre-test  (real 
targets)  was  described  by  a  slope  of  0.93  and  an  intercept  of  -3.1  cm  with  a  squared  correla¬ 
tion  coefficient  1.00. 

The  difference  between  the  slopes  of  the  regression  lines  of  the  pre-test  and  the  subjective 
calibration  test  in  the  horizontal  dimension  was  not  significant  [t(  10) =2.02,  p>0.05].  Both 
intercepts  were  significantly  smaller  than  zero,  t(5)  =  19.01  and  t(5)  =  14.64,  respectively, 
p<  0.001.  This  indicates  a  systematic  tendency  to  point  too  far  to  the  left  with  a  constant 
factor.  Because  both  the  virtual  and  real  world  show  the  same  effect,  it  can  be  excluded  that 
this  effect  is  caused  by  significant  distortion  of  the  VE  system  in  the  horizontal  dimension. 

Vertical  dimension.  In  the  vertical  dimension  regression  lines  of  the  pre-test  had  a  slope  of 
0.86  and  an  intercept  value  of  -1.81  cm  with  a  squared  correlation  coefficient  p2  =  1.00. 
These  regression  lines  reflect  possible  open-loop  pointing  errors.  Differences  between  pre-test 
(real  targets)  and  subjective  calibration  results  (virtual  targets)  reflect  distortions  of  virtual 
space.  Only  the  difference  between  the  intercepts  (not  between  the  slopes)  for  pre  and  subjec¬ 
tive  calibration  tests  was  significant  [t(10)=3.43,  p<0.01]  indicating  that  participants  pointed 
above  the  target  positions  in  virtual  space.  Target  height  appears  to  be  visually  overestimated. 
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3.3  Combining  pre-  and  post-tests:  the  after-effect 

For  each  target  position  the  after-effect  was  determined  by  taking  the  difference  between  the 
post-  and  pre-test.  In  the  lateral  direction,  this  difference  was  signed  positive  when  it  was  in 
the  opposite  direction  of  the  displacement  of  the  virtual  hand.  The  mean  after-effects  in  mono- 
and  stereoscopic  conditions  are  shown  in  Figure  5. 
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Fig.  5  Mean  after-effects  under  stereoscopic  and  monoscopic  conditions  for 
each  dimension.  The  whiskers  indicate  a  magnitude  that  equals  1.96  times  the 
standard  error.  When  the  whiskers  do  not  touch  the  zero  base-line,  the  mean 
after-effects  differ  significantly  from  zero  (p<0.05). 


The  after-effects  in  the  vertical  dimension  did  not  differ  significantly  from  zero. 

The  after-effect  in  the  horizontal  (lateral)  dimension  differed  significantly  form  zero  by  95  % 
confidence  intervals.  The  difference  between  monoscopic  and  stereoscopic  immersion  was  not 
significant  [t(7)=1.35,  p=0.22].  After-effects  along  the  horizontal  dimension  are  most  likely 
to  be  caused  by  the  lateral  displacement  of  the  virtual  hand.  People  compensate  for  a  virtual 
hand  that  is  continuously  perceived  left  of  the  real  hand  position  by  adapting  their  eye-hand 
coordination.  This  compensation  remains  active  after  returning  to  the  real  world  for  the  time 
that  their  real  hand  is  invisible.  The  after-effect  is  systematically  in  the  opposite  direction  of 
the  lateral  displacement  and  is  about  20%  of  the  magnitude  of  the  lateral  displacement.  Figure 
6  shows  the  lateral  after-effects  for  different  target  positions.  Interestingly,  participants 
pointed  2  cm  left  of  the  target  positions  during  the  pre-test.  This  must  have  been  due  to  open- 
loop  pointing  error.  During  the  post-test  after  a  lateral  shift  to  the  left  during  immersion,  the 
after-effect  resulted  in  a  shift  of  about  2  cm  on  average  to  the  right.  When  the  post-test  was 
carried  out  after  immersion  with  a  lateral  shift  to  the  right,  the  pointed  positions  during  the 
post-test  were  about  2  cm  to  the  left  of  the  pre-test  positions. 
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Fig.  6  Lateral  after-effects  (along  the  horizontal  dimension)  for  different  target  posi¬ 
tions.  Targets  are  shown  as  projections  in  the  frontal  plane  (orthogonal  to  the  viewing 
direction).  Filled  symbols  indicate  the  positions  pointed  to  during  the  pre-test.  Open 
symbols  show  the  pointed  positions  during  the  post-tests.  The  difference  in  positions  of 
a  target  between  pre-  and  post-test  (caused  by  immersion  in  VE)  is  called  the  after¬ 
effect. 


In  the  depth  dimension,  after-effects  in  the  stereoscopic  condition  were  significantly  larger 
than  those  in  the  monoscopic  condition  [t(7)=4.91,  p<0.05].  It  was  only  in  the  stereoscopic 
condition,  that  the  after-effect  (4.3  cm)  differed  significantly  from  zero  by  95%  confidence 
intervals.  This  after-effect  shows  that  during  the  post-test  participants  pointed  to  positions 
behind  the  actual  position  of  the  real  targets.  This  might  have  been  due  to  exposure  to  a 
virtual  world  in  which  the  virtual  hand  was  visually  perceived  closer  to  the  body  than  the 
actual  real  hand  position.  For  example.  Figure  4  shows  a  perceptual  underestimation  of  target 
position  in  VE  of  (subjective  calibration  versus  pre-test)  of  14  cm  at  a  target  distance  of  45 
cm.  Participants  therefore,  had  to  position  their  real  hand  behind  a  virtual  target  in  order  to 
touch  the  target  with  their  virtual  hand.  As  a  consequence,  after  returning  to  the  real  world, 
participants  kept  pointing  behind  the  acmal  position  of  real  targets  when  their  hand  was 
invisible.  In  conclusion,  the  after-effect  in  the  depth  dimension  is  consistent  with  the  under¬ 
estimation  of  depth  in  the  virtual  world  as  shown  in  the  subjective  calibration  experiment. 


3.4  The  dependency  of  after-effects  on  target  depth 

The  subjective  calibration  experiment  clearly  showed  that  the  perceived  distance  (in  depth)  of 
targets  in  virtual  space  is  systematically  underestimated.  In  fact,  the  linear  relation  between 
indicated  positions  in  virtual  space  and  acmal  positions  has  a  regression  coefficient  of  0.55. 
This  suggests  that  the  perceived  virtual  space  is  scaled  or  minified  with  respect  to  the  geomet¬ 
ric  world  it  represents.  If  we  hypothesize  that  after-effects  are  caused  by  percepmal  deforma- 
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tions  of  virtual  space,  it  is  interesting  to  check  how  the  after-effects  observed  depend  on  target 
depth.  Does  the  after-effect  scale  with  target  depth  or  is  it  constant? 

For  this  purpose  we  have  plotted  the  after-effect  as  a  function  of  target  depth  (see  Figure  7). 
The  regression  lines  are  also  shown  in  Figure  7,  The  regression  line  for  the  pre-test  has  a 
slope  of  1.07  and  an  intercept  of  -4.29  cm  (r2=0.98).  For  the  post-test  we  find  a  regression 
line  with  a  slope  of  1.10  and  an  intercept  of  -1.58  cm  (r^= 0.98). 


target  depth  (cm) 


Fig.  7  Indicated  (pointed)  target  distances  along  the  viewing  axis  (depth)  plotted 
against  the  actual  distance  of  the  real  targets  for  both  pre-  and  post-tests.  Only 
those  pre-  and  post-tests  preceding  and  following  stereo  exposures  to  VE  are 
included. 


The  slopes  of  the  regression  lines  of  pre-  and  post-test  do  not  significantly  differ  [t(10)=0.44, 
p>0.5],  although  the  slope  of  the  pre-test  is  slightly  larger.  The  intercepts  did  not  differ 
significantly  either  [t(10)=0.77,  p<0.5].  The  magnitude  of  the  after-effect  is  4.3  cm  on 
average.  Due  to  the  small  after-effect  and  the  limited  range  of  target  positions  tested  it  is  hard 
to  distinguish  a  shift  of  regression  lines  from  a  scaling  effect.  We  carried  out  an  ANOVA 
analysis  (within  subjects)  with  two  independent  variables.  The  first  variable  was  the  type  of 
test  (pre-test  versus  post-test)  and  the  second  was  the  target  distance  (33.1  versus  55.2  cm 
from  the  eye).  A  significant  interaction  effect  would  indicate  a  difference  between  slopes  of 
pre-  and  post-test. 

The  non-significant  interaction  effect  found  (p==0.13)  suggests  that  the  regression  lines  are 
shifted  and  thus  that  adaption  does  not  scale  with  target  depth  but  is  constant. 
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To  further  reveal  some  characteristics  of  the  adaptation  process  we  have  studied  the  depend¬ 
ency  between  magnitudes  of  lateral  after-effects  and  after-effects  in  depth.  If  a  single  process 
would  generate  both  types  of  after-effects,  the  magnitudes  of  both  effects  are  expected  to  co¬ 
vary  within  a  subject  (large  magnitudes  along  both  axes  for  sensitive  participants,  small 
magnitudes  for  insensitive  participants). 

We  have,  therefore,  correlated  the  individual  magnitudes  along  the  two  axes  (horizontal  and 
depth)  under  stereo  conditions.  Pearson’s  r  was  0.02.  This  zero  correlation  effect  suggests 
that  the  adaptation  in  depth  and  the  adaptation  along  the  horizontal  axes  are  caused  by  inde¬ 
pendent  processes. 


3.5  The  effect  of  a  displaced  virtual  hand  on  manipulation  performance 

In  this  section  we  check  whether  the  displacement  of  the  virtual  hand  influences  manipulation 
performances  as  reflected  in  time  it  takes  to  grasp,  pitch,  roll  and  position  the  dice  and  the 
accuracy  of  positioning  are  influenced  by  displacing  the  virtual  hand.  In  Werkhoven  and 
Groen  (1997)  these  performance  indicators  were  measured  for  identical  tasks,  but  with  a 
virtual  hand  that  was  not  laterally  displaced  as  in  our  current  experiments. 

In  order  to  be  able  to  compare  the  results  of  the  present  experiment  with  the  results  of  Werk¬ 
hoven  and  Groen  (1997)  it  was  necessary  to  consider  only  the  trials  that  were  carried  out 
under  identical  conditions  in  both  experiments.  Grasping  and  rotation  completion  times  of 
both  experiments  were  therefore  included  only  when  the  die  had  a  size  of  10  cm  and  when  it 
was  initially  presented  on  the  right-hand  side  of  the  participants.  The  grasping  times  for  the 
positioning  task  of  the  present  experiment  were  not  taken  into  the  analysis  because  these 
grasping  times  included  the  time  taken  by  participants  to  explore  the  position  of  the  imaginary 
target  square.  Further,  for  grasping  and  rotating,  the  first  eight  trials  in  the  present  experi¬ 
ment  were  considered  practice  trials  and  not  taken  into  the  analysis. 

Positioning  errors.  The  positioning  error  is  defined  as  the  Euclidean  distance  between  the 
indicated  position  and  the  target  position.  Separate  t-tests  for  comparing  the  means  of  posi¬ 
tioning  errors  between  conditions  with  and  without  a  laterally  displaced  virtual  hand  revealed 
no  significant  differences  in  positioning  errors.  For  the  group  with  a  shifted  virtual  hand,  the 
mean  positioning  error  was  7.6  cm  in  the  stereo  condition  and  11.1  in  the  mono  condition. 
For  the  group  with  an  aligned  virtual  hand  this  was  8.9  cm  and  13.9  cm,  respectively,  the 
difference  not  being  significant  [t(10)  =  .85,  p>0.40  and  t(10)  =  1.65,  p>0.10,  respectively]. 

Completion  times.  We  also  compared  the  completion  times  of  manipulation  actions  such  as 
grasp,  pitch,  roll  and  positioning  between  the  groups  with  aligned  and  with  shifted  virtual 
hands  (Figure  8).  Again,  grasping  times  are  considered  for  the  rotation  tasks  only,  not  for 
positioning  tasks. 


27 


(/) 

0 


c 

o 

0 

Q. 


E 


o 

o 


stereo  mono 


Fig.  8  Mean  completion  times  of  grasping,  pitching  and  rolling  tasks  for  the 
group  with  an  aligned  virtual  hand  and  the  group  with  a  laterally  shifted  virmal 
hand,  in  both  stereoscopic  and  monoscopic  conditions. 


No  significant  differences  between  aligned  and  non-aligned  conditions  were  found.  It  should 
be  noted  that  positioning  completion  times  of  the  present  experiment  include  the  time  needed 
to  lift  the  cube  which  may  take  longer  compared  to  the  previous  manipulation  experiments 
(Werkhoven  &  Groen,  1997)  where  the  cube  was  already  ‘in  the  air’  when  positioning 
started.  The  completion  times  needed  for  positioning  only  (not  lifting)  may  therefore  even  be 
a  little  shorter  for  the  present  experiment  (shifted  virtual  hand  conditions)  than  for  the  previ¬ 
ous  experiment  (aligned  virtual  hand). 


4  GENERAL  DISCUSSION 

4.1  Perceptual  deformations  of  virtual  space  in  our  VE  system 

Perceptual  deformations  were  revealed  by  the  subjective  calibration  test  after  correction  for 
the  open-loop  pointing  errors  as  measured  in  the  pre-test.  Therefore,  slopes  and  intercepts  of 
regression  lines  of  calibration  and  pre-test  were  compared. 
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Horizontal  dimension 

A  comparison  of  the  subjective  calibration  test  and  the  pre-test  revealed  no  significant  percep¬ 
tual  distortion  of  virtual  space  along  the  horizontal  dimension.  Our  data  suggest,  however, 
that  there  might  be  a  slight  but  systematic  minification  in  the  lateral  direction.  Combining  this 
with  the  programmed  shift  of  10  virtual  cm,  we  can  conclude  that  in  the  horizontal  dimension, 
the  hand  was  perceived  maximally  10  cm  to  the  right  or  to  the  left. 

Depth  (viewing  axis) 

Distances  along  the  viewing  axis  were  perceived  closer  by  a  scale  factor  of  almost  50%,  that 
is,  when  disparity  was  present.  Under-estimations  of  depth  are  not  exclusively  explained  by 
low-quality  HMD  optics  or  disparity  problems.  An  alternative  explanation  is  in  terms  of 
monoscopic  cues  that  are  incorrectly  interpreted.  For  example,  the  distance  of  a  cube  can  be 
estimated  from  its  projected  retinal  size  when  its  objective  size  is  known.  We  call  this  the  size 
cue.  Werkhoven  and  Groen  (1997)  found  that  the  retinal  size  of  virtual  objects  was  a  strong 
distance  cue  when  the  layout  of  visual  space  was  ambiguous,  in  particular  under  monoscopic 
conditions  and  when  head-movements  are  restricted  such  that  motion  parallax  cues  were 
ineffective.  However,  also  under  stereoscopic  conditions  and  in  the  presence  of  motion 
parallax  cues,  effects  of  size  cues  were  observed.  When  objective  knowledge  about  the  actual 
cube  size  is  not  available  and  the  size  is  estimated  incorrectly  the  perceived  distance  of  the 
cube  may  be  affected.  For  example,  when  the  objective  size  of  the  virtual  cubes  is  estimated 
twice  as  small  as  the  objective  size,  the  cube  may  appear  twice  as  near.  Relying  on  such  size 
cues  effectively  scales  the  range  of  perceived  distances  of  the  virmal  targets  (cubes)  used  in 
the  subjective  calibration  tests.  Of  course,  one  may  wonder  why  the  objective  size  of  the 
target  cubes  was  systematically  under-estimated  (they  appeared  closer  than  they  were)  and  not 
over-estimated. 

Vertical  dimension 

A  comparison  of  pre-test  and  subjective  calibration  results  for  the  vertical  dimension  has 
shown  a  slight  but  systematic  over-estimation  of  height  in  virtual  space  of  about  1  cm.  Conse¬ 
quently,  the  height  of  the  virtual  hand  will  be  perceived  above  the  real  hand  position.  In  the 
next  paragraph,  where  we  compare  post-  and  pre-tests,  we  will  discuss  this  observation  in 
relation  to  the  after-effects  found. 


4.2  After-effects 
4.2.1  Horizontal  dimension 

A  small  but  reliable  after-effect  was  found  in  the  opposite  direction  of  the  displacement  of  the 
virtual  hand.  The  average  effect  amounted  to  20%  of  the  theoretical  maximum  of  10  cm.  Data 


29 


suggests  that  this  was  less  in  the  monoscopic  condition,  the  difference  being  not  significant, 
however.  The  direction  of  the  after-effects  found  for  different  targets  correlated  perfectly  with 
the  directions  of  the  intentional  shifts  (left/right  randomized)  indicating  that  the  lateral  shifts 
have  indeed  caused  the  after-effects. 

What  to  expect  based  on  literature?  The  20%  negative  after-effect  corresponds  to  the  percent¬ 
age  found  by  Held  and  Durlach  (1991)  under  a  delay  of  60  ms  between  hand  movements  and 
visual  feedback.  In  their  experiment,  the  hand  was  represented  by  a  luminous  dot.  In  prism- 
adaptation  experiments,  after-effects  for  relatively  short  exposure  periods  normally  never 
exceed  70%  (Welch,  1978).  It  is  possible  that  the  magnitude  of  the  after-effect  found  in  the 
present  study  was  negatively  influenced  by  a  number  of  factors.  In  the  period  between  immer¬ 
sion  in  the  virtual  environment  and  the  post-test  spontaneous  decay  of  adaptation  might  have 
taken  place.  Adaptation  after-effects  disappear  when  information  about  the  real  hand  position 
is  present  (Welch,  1978).  When  the  experimenter  prepared  the  post-test  and  handed  the 
position  sensor,  proprioceptive  and  auditive  spatial  information  about  the  hand  might  have 
started  the  decay  process.  Furthermore,  after-effects  are  found  to  decrease  as  the  situation  of 
exposure  to  sensory  discordance  and  post-test  are  less  resembling.  For  instance,  in  a  prism 
adaptation  experiment,  Ulharik  and  Canon  (1970)  found  a  smaller  effect  when  the  prism 
goggles  were  removed  than  when  goggles  without  prisms  were  worn  during  pre-  and  post¬ 
test.  In  the  present  study,  the  situations  of  exposure  and  both  pre-  and  post-test  differed  in  a 
great  number  of  aspects. 

Where  does  adaptation  take  place?  It  is  not  clear  from  this  study  what  the  locus  of  adaptation 
was.  Adaptation  can  involve  a  proprioceptive  shift,  (a  recalibration  of  felt  limb  position)  or  a 
visual  shift  (a  recalibration  of  visual  perception)  or  a  combination  of  both.  This  experiment 
measured  the  total  shift.  Welch  (1978)  states  that  the  least  attended  modality  is  more  likely  to 
recalibrate.  In  the  present  exposure  task,  visual  information  was  surely  the  most  important.  It 
is  therefore  likely  that  the  proprioceptive  shift  was  the  main  contributor  to  the  negative  after¬ 
effect. 

4.2.2  Depth  dimension 

A  strong  after-effect  of  more  than  4.0  cm  was  found  in  the  depth  dimension.  Apparently, 
under  the  stereoscopic  condition,  depth  was  sufficiently  well-specified  to  invoke  adaptation  in 
the  depth  dimension.  Participants  adapted  their  eye-hand  coordination  to  a  new  condition  in 
the  virtual  environment  under  which  the  virtual  hand  was  perceived  closer  than  the  real  hand. 
To  solve  this  inconsistency,  they  learned  to  rely  on  the  visually  perceived  position  of  the 
virtual  hand  for  manipulation  purposes  and  ignore  the  proprioceptively  perceived  real  hand 
position.  This  adaptation  effect  lasted  even  after  returning  in  the  real  world. 

Disparity  and  accommodation.  The  positions  pointed  to  with  the  real  hand  during  the  post¬ 
tests  were  systematically  further  away  than  the  actual  target  position.  This  suggests  that  during 
the  adaptation  phase  in  the  virtual  world  the  distance  of  targets  and  of  the  virtual  hand  were 
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under-estimated  relative  to  the  geometric  world  that  was  represented.  The  absence  of  an  after¬ 
effect  in  the  monoscopic  condition  suggests  that  depth  cues  as  convergence  and  disparity  are 
involved  in  the  recalibration  of  eye-hand  coordination.  Accommodation  was  fixed  in  both 
conditions  and  could  not  possibly  have  been  adapted. 

Eye  muscle  potentiation.  A  completely  different  explanation  for  the  difference  in  pointing 
performance  between  pre-  and  post-test  is  the  possible  occurrence  of  eye  muscle  potentiation. 
After  prolonged  efferent  signals,  muscles  show  a  reflexive  contraction  for  a  period  after  the 
efferent  signal  has  stopped.  This  effect  has  been  demonstrated  for  eye  muscles  after  sustained 
convergence  and  found  to  influence  distance  perception  (Paap  &  Ebenholtz,  1977).  The 
decoupling  between  accommodation  and  convergence  while  looking  at  displays  could  have 
caused  abnormal  efferent  signals  to  diverge  or  converge  the  eyes.  Returning  to  the  real  world, 
a  divergence  may  have  persisted  possibly  causing  an  over-estimation  of  distance.  As  a  result 
participants  would  point  behind  the  target  during  post-test.  Thus,  eye  muscle  potentiation  is  a 
possible  explanation  alternative  to  the  adaptive  re-alignment  hypothesis  for  the  observed  after¬ 
effects  in  depth  (not  for  the  other  dimensions!). 

Have  other  researchers  found  unique  evidence  for  eye  muscle  potentiation  or  adaptive  re¬ 
alignment  in  the  depth  dimension?  Rolland  et  al.  (1995)  recently  studied  adaptation  effects  and 
shifted  the  virtual  hand  across  a  distance  of  16.5  cm  along  the  viewing  axis  towards  the 
participants.  They  observed  after-effects  with  a  magnitude  of  4  cm  which  corresponds  nicely 
with  the  magnitude  of  the  after-effect  observed  in  our  experiment.  Has  Rolland’s  after-effect 
been  caused  by  adaptive  re-alignment?  Perhaps  yes.  However,  in  that  case  we  have  to  assume 
a  psychological  limit  of  adaptation  (saturation)  because  we  notice  similar  after-effects  (4cm) 
for  Rolland’s  as  well  as  our  experiments  for  different  discrepant  positions  of  the  virtual  hand 
in  depth.  Alternatively,  however,  their  after-effect  is  also,  at  least  in  part,  caused  by  the 
prolonged  use  of  a  stereoscopic  HMD  (e.g.  eye-muscle  potentiation). 

To  test  the  eye  muscle  potentiation  hypothesis  in  our  experiments,  we  have  informally  tested 
participants  before  and  after  immersion  in  a  ‘handless’  virtual  environment,  where  they  did 
the  same  task  using  mouse-cursor  control.  First  results  indeed  show  similar  pointing  errors  as 
observed  in  our  adaptation  experiments  which  supports  the  eye-muscle  potentiation  hypothesis 
for  the  depth  dimension. 

Independent  processes.  A  zero  correlation  was  found  between  after-effects  in  the  horizontal 
and  those  in  the  depth  dimension.  This  indicates  that  after-effects  in  the  two  directions  are 
mutually  independent.  We  hypothesize  that  the  after-effects  in  the  lateral  horizontal  directions 
have  been  caused  by  a  recalibration  of  the  eye-hand  coordination  (adaptation)  whereas  the 
after-effects  in  depth  have  been  caused  by  continuing  abnormal  efferent  signals  after  immer¬ 
sion  (eye-muscle  potentiation).  In  our  case,  a  visual  calibration  could  mean  a  recalibration 
between  disparity  or  convergence  or  both  and  the  perceived  distance.  Such  recalibration  of 
disparity  and  perceived  distance  has  been  found  in  the  past  (Wallach  et  al.,  1963).  Further, 
Wallach  and  Smith  (1972)  found  a  visual  recalibration  as  well  as  a  proprioceptive  recalibra- 
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tion  of  eye-hand  coordination  after  exposure  to  a  condition  where  convergence  conflicted  with 
accommodation  and  felt  hand  position. 

Plasticity.  It  is  remarkable  that  the  difference  in  pointing  performance  between  pre-  and  post¬ 
tests  in  the  depth  dimension  involved  a  constant  shift,  rather  than  a  scale  factor.  Given  the 
perceptual  distortion  (a  depth  compression  of  almost  50%)  as  revealed  by  the  subjective 
calibration  tests,  a  scale  factor  between  one  and  two  was  expected  to  describe  the  relation 
between  real  and  judged  distance  during  the  post-tests. 

If  eye  muscle  potentiation  is  the  right  explanation,  one  may  ask  why  eye  muscle  potentiation 
results  in  a  shift  of  the  perceived  virtual  space  rather  than  in  a  scaling. 

If  recalibration  of  eye-hand  coordination  is  the  right  explanation,  one  may  wonder  why  the 
brain  is  not  capable  of  recalibrations  other  than  shifts  of  virtual  space.  It  is  not  clear  from  the 
literature  which  spatial  mapping  functions  comprise  the  plasticity  of  the  brain.  There  is 
evidence  that  only  linear  functions  can  be  used  (Bedford,  1993).  Clearly,  conclusions  about 
the  present  finding  of  changes  in  distance  perception  have  to  be  drawn  carefully.  Further 
research  is  needed  to  reveal  the  exact  nature  of  the  observed  after-effects. 

4.2.3  Vertical  dimension 

The  comparison  of  post-  and  pre-tests  along  the  vertical  dimension  revealed  an  insignificant 
but  systematic  after-effect  in  both  monoscopic  and  stereoscopic  conditions.  Participants 
pointed  about  1  cm  higher  in  the  post-test  after  immersion  in  VE  than  in  the  pre-test  before 
immersion. 

This  after-effect  cannot  be  a  result  of  an  adaptation  of  the  eye-hand  coordination  to  a  VE 
immersion  in  which  the  virtual  hand  was  systematically  perceived  beneath  the  real  hand.  This 
has  been  proven  by  comparing  the  results  of  subjective  calibration  and  pre-tests  indicating  that 
the  virtual  hand  was  perceived  above  the  real  hand.  Thus,  we  should  call  this  after-effect  a 
positive  after-effect  in  contrast  to  the  negative  after-effects  found  in  other  dimensions. 

Rolland  et  al.  (1995)  also  studied  adaptation  effects  after  immersion  in  an  augmented  reality 
system.  In  their  experiments  head-mounted  cameras  were  positioned  above  the  eyes.  Conse¬ 
quently  the  hand  was  perceived  6.2  cm  beneath  the  real  hand  position.  They  observed  an 
after-effect  with  a  magnitude  of  1  cm  in  the  opposite  direction  of  the  vertical  displacement  of 
the  perceived  hand  and  called  this  a  negative  after-effect. 

First,  from  their  article,  it  is  not  clear  whether  this  after-effect  was  significantly  different  from 
zero.  Second,  if  it  is  significant,  it  is  not  evident  that  the  vertical  after-effect  is  caused  by  the 
vertical  displacement  of  the  perceived  ion  during  immersion.  We  say  this  because  we  found  a 
similar  after-effect  after  immersion  with  a  virtual  hand  that  was  perceived  1  cm  above  the  real 
hand  position  instead  of  6.2  cm  beneath  the  real  hand  position  as  in  Rolland’s  experiments.  So 
it  seems  that  the  same  after-effect  can  be  obtained  with  opposite  displacements  of  the  per¬ 
ceived  hand  suggesting  that  an  explanation  in  terms  of  adaptation  of  the  visuo-motor  pathway 
is  not  likely. 
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4.3  Simulation  fidelity 

After-effects  in  lateral  direction  are  a  measure  of  the  simulation  fidelity.  We  found  significant 
after-effects  under  stereoscopic  conditions,  but  not  under  monoscopic  conditions.  One  may 
therefore  say  that  the  simulation  fidelity  decreases  under  monoscopic  conditions. 

The  occurrence  of  negative  after-effects  in  the  lateral  direction  under  stereoscopic  conditions 
shows  lower  level  parameter  adjustment  in  eye-hand  coordination  (Redding  &  Wallace, 
1996).  This  is  promising  for  those  interested  in  using  virtual  hands  to  acquire  visuo-motor 
skills.  Skilled  behavior  is  characterized  by  open-loop  control  and  replacing  visual  feedback  by 
proprioceptive  feedback.  According  to  the  theory  of  motor  skills,  motor  programs  are  stored 
in  the  brain  in  a  general  form.  At  a  lower  level,  parameters  are  filled  in  to  fine  tune  the  motor 
program  to  the  needs  of  the  specific  situation  (Welch,  1978).  Because  adaptation  appears  to 
take  place  at  a  low  level  and  their  after-effects  are  found,  it  is  likely  that  the  motor  skill 
strategy  at  a  higher  level  is  not  affected  and  can  be  transferred  to  real  world  situations. 
Further  research  should  point  out  the  limits  of  the  spatial  and  temporal  discrepancies  in  VE 
systems  allowing  a  transfer  of  visuo-motor  learning.  Adaptation  to  discrepant  visual  informa¬ 
tion  from  the  fingers  also  has  to  be  studied  in  this  context. 


4.4  Comparison  between  non-aligned  and  aligned  virtual  hand  control 

The  data  from  the  present  manipulation  performance  task  with  discrepant  lateral  virtual  hand 
positions  did  not  reveal  any  decline  in  speed  or  accuracy  in  the  present  experiment  compared 
to  the  data  obtained  by  Werkhoven  and  Groen  (1997)  for  aligned  virtual  hand  control.  For  the 
tasks  used,  which  are  typical  grasping,  rotating  and  positioning  tasks  with  small  objects,  a 
lateral  difference  of  10  cm  does  not  seem  to  influence  performance. 

Rolland  et  al.  (1995)  found  a  decrease  in  a  manual  pegboard  task,  when  hand  position  was 
shifted  vertically  and  in  depth,  compared  to  performance  on  the  same  task  in  the  real  world. 
They  attribute  this  performance  decrease  to  the  intersensory  conflict  (mis-alignment),  although 
they  admit  that  the  poor  HMD  resolution  might  play  a  role  (estimated  to  be  8  arcmin,  see 
Edwards,  Rolland  &  Keller,  1993).  Our  experiments  showed  that  a  discrepancy  between 
perceived  and  real  hand  position  did  not  measurably  decrease  manipulation  performance 
compared  to  aligned  conditions  though  both  conditions  were  tested  in  the  virtual  world  using  a 
HMD  with  a  resolution  of  4.4  arcmin.  Thus,  our  findings  suggest  that  the  decrease  compared 
to  the  real  world  found  by  Rolland  et  al.  is  not  likely  to  be  caused  by  mis-alignments  of  the 
virtual  hand  but  rather  by  low  HMD  resolution,  perhaps  combined  with  the  absence  of 
oculomotor  cues. 

One  can  not  exclude,  however,  that  manipulation  performance  is  similar  in  laterally  non- 
aligned  and  aligned  conditions  because  the  virtual  environment  tested  already  showed  an 
inherent  perceptual  distortion  in  depth.  This  distortion  in  depth  may  have  affected  manipula- 
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tion  performance  under  laterally  aligned  as  well  as  laterally  non-aligned  conditions.  The  mis¬ 
alignment  in  depth  may  even  have  dominated  the  effects  of  lateral  mis-alignment.  To  isolate 
the  effects  of  lateral  mis-alignments  the  experiments  should  be  carried  out  in  a  perfectly 
calibrated  virtual  environment.  For  now,  we  conclude  that  the  misaligranents  of  current  VE 
systems  as  studied  in  our  experiments  do  not  affect  manipulation  performance. 


5  CONCLUSIONS 

•  After-effects  of  adapted  eye-hand  coordination  during  VE  immersion  with  percepmally 
misaligned  virtual  hand  position  were  revealed  by  the  outcome  of  pointing  tasks  during 
subjective  calibration,  pre-  and  post-tests.  Pre-tests  revealed  open-loop  pointing  errors  that 
serve  as  a  base-line  for  interpreting  the  results  of  pointing  tasks  in  VE  (subjective  calibra¬ 
tion)  and  during  post-tests  (showing  after-effects).  A  comparison  of  subjective  calibration 
and  pre-tests  showed  perceptual  distortions  of  virtual  space.  A  comparison  of  post-tests  and 
pre-tests  showed  after-effects  of  adaptation  of  eye-hand  coordination  due  to  VE  immersion. 

•  After-effects  in  the  lateral  direction  were  found  opposite  to  the  shift  of  virtual  hand  posi¬ 
tion  during  immersion.  This  negative  after-effect  was  significant  under  stereoscopic 
conditions  and  had  a  magnitude  of  20%  of  the  lateral  shift.  Under  monoscopic  conditions 
no  significant  after-effects  were  found. 

•  After-effects  found  for  the  depth  dimension  were  opposite  to  the  50%  perceptual  minifi- 
cation  of  virtual  space  as  shown  by  the  subjective  calibration  test.  However,  this  negative 
after-effect  did  not  scale  with  the  target  distance  for  pointing.  Instead,  the  after-effect  was 
constant  with  a  magnitude  of  4  cm.  This  after-effect  is  likely  to  be  caused  by  either  percep¬ 
tual  distortions  of  visual  space  due  to  HMD  optics,  or  to  eye-muscle  potentiation  just  after 
immersion.  Further  studies  are  needed  to  discriminate  between  possible  causes. 

•  Manipulation  performance  was  not  affected  by  lateral  shifts  of  the  virtual  hand.  Even  when 
the  virtual  hand  was  perceived  at  another  position  than  the  real  hand,  the  completion  time 
and  accuracy  of  grasping  and  positioning  tasks  did  not  significantly  differ  from  identical 
tasks  under  aligned  conditions. 

•  The  occurrence  of  negative  after-effects  in  lateral  direction  is  a  proof  of  lower  level 
parameter  adjustment  in  eye-hand  coordination.  This  is  promising  for  those  interested  in 
using  virtual  hands  to  acquire  visuo-motor  skills.  Acquired  skills  in  VE  are  likely  to 
transfer  to  the  real  world. 
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